Efficient Data-Sensitive Techniques for Parallel Retrieval of Keyword-indexed Information

Keyword based search of data, such as documents, maps, images, audio and video data, is an everyday activity for many millions of people with myriad uses, e.g., scientific computing, digital libraries, the web, catalogs, geographical information systems, music servers, etc. In this paper, we present several declustering algorithms based on existing similarity measures as well as their generalizations. Experiments show that the new declustering methods are indeed more efficient in declustering times and close in terms of parallel query times than quadratic declustering methods based on existing similarity measures. Our declustering algorithms are sublinear in the number of comparisons and scalable with increasing data and disks. The new methods are also capable of handling streaming data quite efficiently. We present some negative results on random and profile-based sampling. Although the new sampling strategies based on profiles of the documents outperform the old sampling strategies, which do not use a profile, they are still worse than random and round-robin. Further testing is required to confirm the sampling results. 1 Research supported in part by NSF grant CCF 0306475.